

THROUGH ITS EDUCATION INVESTMENTS, THE ANNIE E. CASEY FOUNDATION HELPS ENSURE THAT YOUNG PEOPLE GRADUATE FROM SCHOOL PREPARED TO SUCCEED AS ADULTS 
BY SUPPORTING QUALITY SCHOOL CHOICES AND STRONG SCHOOL/COMMUNITY/FAMILY CONNECTIONS. 
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The core result that the Annie E. Casey Foundation’s education program seeks to achieve 
is that: 

One day, all young people in tough neighborhoods will graduate from school prepared 
to succeed as adults in the worlds of work, family, and citizenship. 

The Foundation has a straightforward “theory of change” about what should happen to 
achieve this result: Quality Choices + Robust Connections = Core Result. If we create 
quality school choices for young people and their families, and if these schools foster robust 
connections among themselves, families, and community institutions, then it is possible 
for all young people to graduate from school prepared to succeed. 

To achieve this core result, Casey’s education portfolio has invested roughly $35 million 
over seven years in two program areas that it believes hold special promise for closing the 
achievement gap and helping low-income children in tough neighborhoods do well in life: 

• Creating quality education choices for low-income families. 

• Building strong connections among schools, communities, and families. 

Through these two investment areas, the Foundation provides program support for start-up, 
planning, and early implementation efforts; technical assistance, capacity building, and 
network development; and policy and data analysis, advocacy, public will building, and 
documentation of lessons learned. 



“[Casey] talks about results in terms of impact, influence, and leverage. It’s a useful frame, and I’ve adopted it.” Jim Shelton, 
Bill and Melinda Gates Foundation 



In addition to making grants, Casey uses program-related investments (PRIs) to advance 
its education program goals. A PRI is an investment in the form of a loan, a loan guarantee, 
a line of credit, a mission-related deposit, an asset purchase, a recoverable grant, or an 
equity investment. The Foundation has used PRIs to attract co-investments in the hard- 
to-finance area of charter facilities to significant effect, helping create thousands of new 
seats for inner-city students. 

The Closing the Achievement Gap series explores Casey’s education investments in detail 
and presents stories, results, and lessons learned. This particular publication describes Caseys 
efforts to develop a flexible but rigorous results measurement system that enables the 
Foundation and its grantees to reflect on practice and then course-correct as necessary to 
achieve the desired result. It also presents a list of selected aggregate results to show the 
Foundation’s own progress in the area of education results measurement. 

Developing a Results Framework that Works for Everyone 

The education program strives to measure its own and its grantees’ progress toward 
achieving the core result while at the same time encouraging innovation, so the results 
measurement system must be flexible as well as clear and somewhat standardized. 




Getting the Most Out of Knowledge Management 



Civic Builders is a nonprofit that finances, designs, 
and develops charter facilities in New York City. 

It has received Casey support in the amount of 
$400,000 over four years to create a knowledge 
management system that includes documenting and 
sharing best practices and lessons learned and 
developing a way to use documentation to improve 
organizational effectiveness. 

At first, Civic Builders wasn’t sure how to use 
documentation to further its work. “Early on, we 
asked ourselves, ‘What does Casey want to see?”' 
said Civic’s co-founder David Umansky. “Now, 
we document projects that are important to our 
own learning. It’s a luxury to get support to do this 
thinking, and we don’t want to waste it." 

For the past few years, Casey funds have supported 
a part-time knowledge manager who performs a 
number of jobs. During focus groups and knowledge- 
sharing gatherings, the knowledge manager listens 



but doesn’t talk. Later, during the debriefing session, 
the knowledge manager is a vocal participant who 
helps staff clarify what happened and distill what 
was most important from the gathering. 

The knowledge manager will then create an appro- 
priate product. If the organization’s board needs 
to use information from the session to make a deci- 
sion, the staff person will create a slide presentation 
and handout. If it’s a best-practices meeting, the 
product might be a summary of ideas shared to send 
out to participants via e-mail. 

Over time, Civic Builders has come to rely on knowl- 
edge management and documentation to trigger 
reflection and improve practice. “Three years ago, 
we might not have sought other money to support 
the knowledge management function,” said Umansky. 
“Now, we are hooked on it. We would definitely 
seek to replace it if Casey support went away.” 



“The results framework gives us tools to talk about results in a complex and meaningful way, and it reminds us that the 
bottom line is, did our work change anything for families and kids?” Mike Kromrey, Metro Organizations for People 



As its results measurement work has evolved over a period of years, Casey has found it useful 
to discuss its results in the categories of impact, influence, and leverage, and it has asked 
its education grantees to do the same. 

• Impact refers to improvements in the success and well-being of children, families, 
or communities directly served by the grantee. 

• Influence refers to positive changes in opinions, practices, or policies as a result of 
the grantee’s work. 

• Leverage refers to new or increased public or private co-investment in Casey- 
supported activities that lead to better results for children and families. 

For the Foundation, the impact, influence, and leverage framework helps sharpen the 
education portfolio’s strategic focus. It guides the selection of new investments, and it 
helps in the evaluation of existing investments and the portfolio as a whole. For grantees, 
the framework helps them refine their focus, better manage and improve their programs, 
and communicate about their successes to others. For both, it ensures that people are using 
a common language to discuss progress toward results. 



Early on, Casey reviewed existing results measurement systems at other foundations and 
organizations. Those that had worked on results measurement emphasized the value of 
using templates to guide data collection, as well as the benefit of creating a system that 
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LESSONS FROM DEVELOPING A RESULTS 
MEASUREMENT FRAMEWORK 



• Clearly articulate your vision of a core result 
and your strategy for achieving it before starting 
to create your results measurement system. 

• Include grantees in the development of the 
performance measurement system to get their 
buy-in and to ensure that the framework is 
relevant. 



• Make sure that grantees understand the logic 
behind the results measurement system, 

and give them the support they need to develop 
capacity to comply with the system. 

• Test and refine your results measurement tools 
until they work for everyone. 

• Building a robust measurement system requires 
years of effort at all levels of a foundation. 



• Support evaluation, documentation of lessons 
learned, internal reflection, and knowledge 
management systems to help grantees learn 
from what they are doing, improve their 
performance, and share their knowledge with 
others. 



“We are focusing our thinking, now, with the end in mind, and we are asking how we can modify our activities to better 
achieve that end.” Bob Pavlik, Institute for the Transformation of Learning, Marquette University 



SELECTED RESULTS 



This partial list of aggregated results to which 

Casey has contributed is derived from results 

listed throughout the Closing the Achievement 

Gap series. 

Impact 

• 81 percent of 6,110 students in 17 individual, 
Casey-supported charter schools have 
improved performance on state math tests. 

• 59 percent of these students have improved 
performance on state reading tests. 

• 57 new charter schools are open in 
Milwaukee, Philadelphia, and Indianapolis. 

• 50 new charter schools with English language 
learner (ELL) focus are open in 16 states. 

• 9 new charter school facilities are open in 
New York City. 

• 193 new charter leaders. 

• 100 charter schools trained in effective ELL 
techniques. 

• 300 DC charter teachers trained in effective 
special education techniques. 

• 300 parents trained each year to understand 
student achievement data and the federal No 
Child Left Behind Act. 



Influence 

• A gradual shift in the national climate of opinion 
regarding school choice policies. 

• A more nuanced view of charters in national 
and state news coverage. 

• Increased national interest in mayoral role in 
public education. 

• Numerous active partnerships in Atlanta, 
Indianapolis, and Washington, DC, in which 
organizations are working together to achieve 
Casey’s core result. 

• Districts in Indianapolis are collaborating with 
charter schools. 

• Supported creation of a model for improving 
student outcomes in a district feeder pattern. 

• Supported creation of effective and replicable 
charter authorizing and accountability systems 
in Indianapolis. 

• Supported creation of an effective and replica- 
ble results reporting system in Indianapolis that 
will work for both charters and traditional pubic 
schools. 

• Supported progress in creating results 
measurement tools for programs that connect 
schools, communities, and families. 

• Supported a rigorous evaluation that will 
provide nuanced data on voucher impact, 
benefits, and challenges. 



• Supported the creation of the first national 
leadership organization for charter schools. 

• Supported the creation of the first national 
research center on charter schools. 

• 6 state charter laws improved. 

• 30+ publications on evidence-based school 
improvement policy and practice. 

• 4 evidence-based improvements in the national 
special education law. 

• 3,700 people who work for improved education 
policy and practice convened each year by 
nine organizations. 

• 700 philanthropists trained each year in 
effective education grantmaking practices. 

Leverage 

• $50 million for charter facilities. 

• $18.4 million in private philanthropic support to 
charter schools in Indianapolis. 

• $4.4 million in private philanthropic support 
invested in the DC voucher program. 

• $1.9 million in public, private, and in-kind 
support for improvement in a district feeder 
pattern in Atlanta. 

• Numerous investments by other national and 
local donors in Casey grantees that are working 
to achieve the Foundation’s core result in 
education. 
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makes it possible to aggregate results. They also suggested that grantees would need hands- 
on support from consultants to develop capacity to use any results measurement tool. 

Casey has included grantees in the development of its results measurement system from the 
beginning, in order to make sure the Foundation asks the right questions and to ensure 
grantee buy-in. During the development process, Casey found that many grantees were 
eager to create a results-based culture within their own organizations. The biggest chal- 
lenge has been to create a data collection template that enables the Foundation to aggregate 
results while giving grantees room to name their own additional performance measures 
and express their individuality. 



“Telling our affiliates that they have to do documentation and ongoing evaluation isn’t enough. We have to provide them 
with materials and templates and teach them how to use them.” Dr. Velma Cobb, National Urban League 



In 2007, Casey drafted a performance measurement guidebook and accompanying spread- 
sheet for education grantees. Currently, the Foundation is field-testing these tools through- 
out the education portfolio. It will continue to adjust and refine the tools based on user 
experiences. 

Encouraging Internal Reflection to Improve Results 

A key component of any results measurement effort is regular reflection on progress toward 
outcomes in order to refine and improve programs and practice. Casey’s education pro- 
gram encourages grantees to reflect on and learn from what they are doing by supporting 
a variety of activities. These activities include setting up knowledge management systems, 
documenting stories and lessons learned, and using evaluation and data to drive and 
refine programming. 

Casey has provided support to a number of grantees, including Civic Builders, NewSchools 
Venture Fund, and New Leaders for New Schools to set up internal knowledge management 
systems. These systems typically include online libraries of all documents created by an 
organization, so staff can easily find and retrieve knowledge, along with documentation 
of lessons learned in the field that is distributed widely outside the organization. 

Organizations usually benefit from having a dedicated knowledge manager to set up and 
manage these online systems, and to train staffers to use them. This person can also sit in 
on meetings, trainings, and other gatherings, participate in subsequent debriefing sessions, 
write up summaries for a variety of internal uses, and help leaders check in on progress 
toward desired results. 

“Casey helped us understand the concept of being a learning organization by encouraging 
us to be intentional about knowledge management,” said Julie Petersen of NewSchools 
Venture Fund. 
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The Foundation has also supported documentation and lessons learned activities that 
encourage internal reflection. “Documentation improves our performance,” said David 
Umansky, co-founder of Civic Builders. “We have greater impact if we do the thinking 
and analyzing that documentation forces us to do. It encourages reflective practice.” 

Casey has found that data and analysis from external evaluations can also help organizations 
learn from what they are doing to improve practice and make better progress toward results. 
For example, Casey’s early support for ongoing evaluation at New Leaders for New Schools 
enabled that organization to align its programmatic work to its ultimate goal of improving 
student achievement in urban schools. 

Sharing Knowledge About What Works 

In another effort to encourage effective results-oriented work, Casey’s education program 
encourages its grantees to share best practices and lessons learned by supporting publications, 
gatherings, and online communities. 

With Casey support, a national network of education philanthropists called Grantmakers 
for Education has created and disseminated a series of case studies about effective grant- 
making practice. 



Casey helped us understand the concept of being a learning organization by encouraging us to be intentional about 
knowledge management.” Julie Petersen, NewSchools Venture Fund 



In Denver, the grassroots community organizing group Metro Organizations for People 
created a lessons learned publication that spread the word about how to mobilize urban 
families to influence local school policy. 

In Washington, DC, Washington Scholarship Fund uses Casey support to create annual 
chronicles and lessons learned publications to describe what has happened, explore chal- 
lenges, and share lessons learned as they create and implement the first federally funded 
voucher program in the country. 

Casey also invests in large knowledge-sharing gatherings for a number of different affinity 
groups, including charter school operators, education philanthropists, potential school 
leaders, people who work with English language learners, and those who specialize in 
after-school programs that support standards-based learning. Some grantees share what 
they learn from their work in small face-to-face meetings. 

Online communities are the next step in knowledge sharing among groups of people with 
overlapping experience and interests. NewSchools Venture Fund has launched an excit- 
ing online forum where people who are working to transform public education through 
charter schools can engage in collective problem-solving, share effective materials and 



“We share what we are learning primarily in face-to-face conversations.” David Umansky Civic Builders 




Using Online Communities to Share Knowledge 



NewSchools Venture Fund (NSVF) is a nonprofit 
venture philanthropy firm that invests in education 
entrepreneurs and performance-driven organizations 
that seek to have a big impact on public education. 

With support from Casey totaling $900,000 over six 
years, NSVF has developed an internal knowledge 
management system and enabled charter school 
leaders and others to share best practices through 
written documents, community-building events, and 
an online community. 

In 2006, the organization launched the NewSchools 
Community, an online forum that allows the NSVF 
network of people it invests in to share expertise 
and resources. The website includes organization 
profiles and contacts, a discussion section, and an 
extensive library of best-practice documents and 
implementation tools for charter schools, including 
curriculum and student assessment materials. 



A total of 400 people currently use the website, 
and any visitor can see that it is a groundbreaking 
resource. Being on the front edge of the online 
community wave does pose some challenges, 
however. NSVF finds that it is proving difficult to 
get network members accustomed to regularly 
uploading new content, even though people in the 
network are constantly generating useful tools and 
documents. “We expected to be more hands-off 
by now, but we’ve had to stay involved,” said Sarah 
Klem, operations analyst and community website 
guru for NSVF. 

The organization is currently developing a set of 
tracking metrics so staff can understand who is 
using the site and how they are using it. They will 
use this data to take the online community to a 
new level of sophistication and utility. 
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approaches, and find contact information for organizations and people that they want to 
follow up with more personally. 

Next Steps for Casey and Results 

Casey’s education program will continue to refine its results measurement framework based 
on individual feedback from grantees that are starting to use it. The Foundation is also 
making plans to convene its education grantees to share their experiences with results 
measurement, internal reflective practice, ongoing evaluation, and knowledge sharing, so 
that they can learn directly from one another. 

As grantees and the Foundation continue to forge a common language to talk about results, 
Casey’s education program will improve its ability to aggregate its results and make the case 
for the most effective ways to improve K-12 public education and help children graduate 
prepared to succeed as adults. 
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Learning to Use Evaluation Data to 
Drive Programs 



New Leaders for New Schools (NLNS) recruits 
talented people with strong leadership potential and 
trains them to be outstanding urban school princi- 
pals. The organization currently has 450 new leaders 
in district and charter schools, including 240 
principals, 100 assistant principals, and 100 new 
leaders in training. 

With Casey support totaling $322,000 over four 
years, NLNS began to develop an ongoing evaluation 
in 2001 . At first, staffers weren’t sure about what 
guestions to ask and why, but the early Casey- 
supported evaluation work ended up being of pivotal 
importance in helping the organization align meas- 
urable goals with daily operations. 

"Now, evaluation has a huge value to the organiza- 
tion, but it took time to make it valuable,” said 
NLNS co-founder Jon Schnur. "The data and 
evaluation people didn't become truly integrated ... 
until we actually started seeing our data, which was 
in 2005.” 



Since then, NLNS has created effective, simple tools 
to help staff report on progress and make sure that 
what they are doing is leading to improved student 
achievement in urban schools. 

Through the evaluation, NLNS has learned that 
students in schools led by its principals are making 
faster gains than students in other schools in the 
public system, but they need to do even better if 
they are to reach the organization’s ambitious 2014 
goals of having at least 300 schools with NLNS 
leaders in place for at least five years and more 
than 90 percent of students proficient in core 
subjects and graduating ready for college. 

"We look at the data to do analysis and figure out 
what is and isn’t working,” said Schnur. “All of our 
conversations are grounded in data now.” 
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